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1.  Introduction 

Recently  the  problem  of  selecting  the  best  one  of  several  binomial  populations 
has  been  studied  from  the  point  of  view  of  different  sampling  rules.  In  this  paper, 
we  compare  some  sequential  procedures  with  and  without  early  elimination.  The 
main  breakdown  is  between  those  using  the  cyclic  play  the  winner  (PWC)  sam¬ 
pling  rule  and  those  using  the  vector  at  a  time  (VT)  sampling  rule. 

The  PWC  rule  orders  the  k  given  populations  at  random  at  the  outset  and 
uses  this  ordering  in  a  cyclic  manner.  After  each  success,  we  sample  from  the 
same  population;  after  each  failure,  we  switch  to  the  next  population  in  the 
ordering  scheme.  After  the  kth  population,  we  complete  the  cycle  by  going  back 
to  the  first  population. 

The  VT  rule  consists  of  taking  k  tuple  observations,  one  component  from  each 
population.  In  a  variation  of  this,  the  cyclic  (VTC)  rule,  we  start  as  in  the  PWC 
rule  by  randomizing  the  order  of  the  populations  and  then  take  one  observation 
from  each  population  using  the  fixed  cyclic  order;  thus,  we  need  not  complete 
the  last  vector  in  the  VTC  rule. 

Both  of  the  above  rules  can  be  modified  as  follows.  Let  the  order  of  the 
populations  sampled  be  n,  T2,  •  •  •  ,  ?r*.  From  the  beginning  of  sampling  ti  to 
the  end  of  sampling  t *,  we  have  gone  through  one  complete  sampling  cycle. 
Our  new  modification  is  to  reorder  the  k  populations  after  each  complete  sam¬ 
pling  cycle;  this  reordering  can  depend  on  the  observed  results.  We  denote  such 
a  modification  of  the  PWC  and  VTC  rules  by  PWO  and  VTO,  respectively. 

Several  papers  dealing  with  the  PW  and  VT  sampling  rules  [6],  [9],  [12],  and 
[13]  consider  termination  rules  based  on  a  fixed  sample  size  or  on  inverse  sam¬ 
pling,  that  is,  we  sample  until  at  least  one  population  reaches  a  fixed  number  of 
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successes.  In  [10],  k  =  2  and  the  termination  rule  is  based  on  the  difference  of 
the  numbers  of  successes.  The  monograph  [1]  deals  mainly  with  YT  sampling 
and  a  stopping  rule  based  on  likelihood  ratios.  (A  summary  of  the  above  work 
can  be  found  in  [11].)  Paulson  [7],  [8]  has  brought  in  early  elimination  tech¬ 
niques,  which  (except  for  [1])  is  not  a  feature  of  the  above  references;  some 
discussion  of  elimination  procedures  does  appear  in  Chapter  9  of  [1]. 

In  this  paper,  we  introduce  a  new  procedure  that  combines  the  likelihood 
approach  to  the  stopping  rule  with  the  PWC  sampling  rule.  In  Section  4,  we 
derive  an  extension  of  the  rule  in  [10]  to  the  case  of  k  populations,  which  con¬ 
tains  the  feature  of  early  elimination.  Empirical  results  for  the  PWO  sampling 
rules  are  obtained  and  analyzed  for  the  inverse  sampling  rule  in  Section  6.  Sec¬ 
tions  3  and  4  deal  with  the  PW  sampling  while  Sections  5  and  7  are  partly 
concerned  with  VT  sampling.  In  Section  5  sequential  techniques  developed  in 
[3]  are  applied  to  Wald’s  sequential  double  dichotomy  formulation  [14],  pro¬ 
ducing  a  binomial  selection  procedure  with  VT  sampling  and  early  elimination 
features. 

In  Section  7,  we  briefly  describe  two  other  VT  rules,  originally  given  in  [1] 
and  [7].  Finally,  we  present  empirical  results  for  all  of  the  above  procedures  in 
Sections  6  and  7  and  make  appropriate  comparisons. 


2.  Notation,  definition,  and  requirement 

Let  pi  denote  the  single  trial  success  for  population  i n  and  let  qt  =  1  —  p<, 
i  -  1,  2,  •  •  •  ,  k.  The  ordered  p  values  are  denoted  by  pm  ^  p[2]  s£  •  •  •  ^  pro- 
For  pi *]  >  pik- 1],  a  correct  selection  (CS)  is  defined  as  the  selection  of  the  popu¬ 
lation  associated  with  p[fc];  for  equality,  either  selection  is  correct.  Let  A  denote 
the  value  of  pw  —  Pik-i).  A  procedure  R  is  said  to  satisfy  the  (A*,  P*)  probability 
(of  a  correct  selection)  requirement  if 

(2.1)  P{CS[R)  ^  P*  whenever  A  ^  A*; 

here  A*  (with  0  <  A*  <  1)  and  P*  (with  l/k  <  P*  <  1)  are  preassigned  con¬ 
stants.  All  the  procedures  discussed  in  this  paper  satisfy  this  common  require¬ 
ment  (2.1). 

Let  Ni  denote  the  sample  size  taken  from  ir and  let  N  denote  the  sum  over  i 
of  these  sample  sizes,  i  —  1,  2,  ■  •  •  ,  k.  Let  denote  the  sample  size  from  the 
population  associated  with  pw,  i  —  1,  2,  •  •  ♦  ,  k.  Then  we  define  our  loss  func¬ 
tion  by 

(2.2)  L  =  £  (pro  -  Pii])N(i) 

1  =  1 

and  the  corresponding  risk  function  by 

k 

risk  =  L  (pro  -  pm)E{Nw}. 

t=i 


(2.3) 
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In  the  applications  that  we  have  in  mind,  the  one  dealing  with  clinical  trials 
is  uppermost,  where  p,  denotes  the  probability  of  a  cure  using  treatment  i.  In 
this  application,  the  primary  concern  is  to  reduce  the  use  of  poorer  treatments. 
The  risk  function  (2.3)  represents  the  expected  number  of  failures  that  could 
have  been  avoided  if  we  had  known  beforehand  which  treatment  (or  population) 
is  best. 

In  addition  to  the  above  risk  function,  we  are  also  interested  in  reducing  the 
expected  total  number  of  observations ,E{N\R}  for  the  procedure  R. 


3.  Likelihood  procedure  for  play  the  winner  sampling 


In  this  section,  we  consider  the  likelihood  procedure  that  is  appropriate  for 
play  the  winner  sampling.  The  case  of  general  k  is  considered  in  Section  3.1  and 
this  is  specialized  to  k  —  2  in  Section  3.2. 

3.1.  Likelihood  rule  for  PW  sampling  with  general  k.  A  likelihood  rule  based 
on  PW  sampling  and  without  early  elimination  can  be  developed  in  a  manner 
similar  to  that  given  in  [1].  We  describe  this  procedure  for  k  =  3  and  specialize 
to  k  =  2  in  Section  3.2;  the  generalization  to  arbitrary  k  is  a  straightforward 
extension  of  the  case  k  —  3. 

Let  Si  ^  Sz  ^  Si  denote  the  current  number  of  successes  from  the  three  popu¬ 
lations  and  let  F,  represent  the  current  number  of  failures  from  the  population 
associated  with  Si,  i  =  1,  2, 3.  If  S3  =  Sz(>  Si),  we  associate  S3  with  the  smaller 
of  the  two  corresponding  F  values;  similarly  for  Si  —  S2  —  S3,  we  assign  S3  to 
the  one  with  the  smallest  F  value.  If  it  is  still  not  determined,  then  we  use 
randomization.  However,  it  is  shown  below  that  for  P*  ^  Yi  our  rule  never  ter¬ 
minates  sampling  when  randomization  is  used.  Let  pm  <  p[2\  ^  pm  be  the 
ordered  (unknown)  probabilities  of  success  on  a  single  trial. 

The  method,  based  on  the  techniques  in  [1],  is  to  write  the  most  likely  of  the 
three  possible  assignments  of  the  pair  (S3,  Ft)  with  the  ordered  p  values  and  to 
stop  sampling  when  the  minimum  (over  that  part  of  the  parameter  space  for 
which  pi 3j  —  pi 2]  ^  A*)  of  the  corresponding  likelihood  ratio  is  at  least  P*.  More 
specifically,  let  the  likelihood  L  (a,  ft,  y)  be  defined  by 


(3.1)  L(a,  0,  y)  =  pfrj(l  -  pu])F"pfi?](l  ^  Vm)Fl,pMl  -  Pm)Fr, 

where  (a,  /3,  7)  is  a  permutation  of  (1,  2,  3).  Let  the  likelihood  ratio  £(3)  be 
defined  by 

/o  oy  p/o\  _  L(h  2,  3)  +  L(2,  1,  3) 

(3-2)  ■e(3) - m^y) - •’ 

where  the  sum  is  over  all  3 !  =  6  possible  permutations.  This  likelihood  ratio 
£(3)  associates  S3  with  pm  and  £(j)  is  defined  similarly  to  (3.2)  and  associates 
Sj  with  pi 3],  j  =  1,  2.  It  is  a  basic  result  in  [1]  (pp.  17  and  18)  that  if  any  proce¬ 
dure  R  has  the  property  at  stopping  time  that 

(3.3)  max  min  £(j)  ^  P*, 
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where  the  minimum  is  over  all  points  in  the  parameter  space  for  which 
Pm  ~  Pi 2]  A*,  then  R  must  satisfy  the  P*  condition  (2.1).  It  is  believed,  also 
shown  in  [1]  (part  1  of  Theorem  6.1.1),  that  this  minimum  in  (3.3)  is  generally 
attained  at  some  generalized  favorable  configuration  (GLF)  in  which 

(3.4)  pm  =  pm  =  p  -  A*, 

where  we  now  use  p  to  designate  p$]  (this  has  not  been  demonstrated).  Assuming 
that  £(3)  yields  the  maximum  in  (3.3)  at  termination  (which  is  also  not  yet 
shown),  it  follows  by  straightforward  algebra  that  we  can  write  (3.3)  in  the  form 


,«  ^  fL(3,  2,  1)  +  Lj 2,  3, 1)  +  L(l,  3,  2)  +  L( 3, 1,  2)1  1  -  P* 

(3  )  A.““il  L(l,  2, 3)  +  L(2, 1, 3)  J=  P* 

Using  (3.1),  we  obtain  for  (3.5)  the  explicit  form 


(3.6) 


max 


i  -p  y*  ,  (P  -  A*\y»  (_ 
-p+A*j  m  p  )  \i 


i  -p  y*\ 

-p -PA*)  J 


< 


where  T{  =  Sz  —  Si  and  Ui  =  F»  —  Fs,  i  -  1,  2.  For  the  case  of  YT  sampling, 
we  note  that  Fj  —  Fa  =  Sz  —  Sj,  j  =  1,2  and  the  stopping  rule  (3.6)  reduces  to 
that  given  in  [1]  and  in  Section  7  below.  For  PW  sampling  the  above  does  not 
hold  and  we  note  that  Fj  —  Ft  can  only  take  the  values  —1,  0,  and  +1.  If 
Fj  —  Fs  -  —1  for  either  j  =  1  or  j  =  2,  then  the  left  side  of  (3.6)  clearly  tends 
to  oo ,  and  hence  the  inequality  cannot  be  satisfied.  Thus,  we  can  state  the  fol¬ 
lowing  stopping  rule. 

Stopping  rule  for  procedure  Rlpw :  stop  sampling  as  soon  as  F9  min  (Fi,  F2) 
and  (3.6)  holds. 

The  terminal  decision  rule  is  to  select  the  population  associated  with  S9.  If 
Sz  —  Si  and  Fz  —  F2  at  stopping  time,  then  we  randomize  between  these  two 
populations  with  probability  Yi  for  each.  Since  P*  >  pi,  we  cannot  terminate 
with  both  St  =  S2  -  Si  and  Fz  —  F2  =  F\.  Moreover,  for  P*  ^  the  value  of 
(1  —  P*)/P*  fS  1  and  the  inequality  (3.6)  cannot  hold  if  Sz  =  S2  and  Fz  —  Ft. 
Hence,  we  will  never  have  to  randomize  in  our  termination  rule  when  P*  ^  H* 

It  should  be  noted  that  the  procedure  Flpw  is  carried  out  by  computing  the 
maximum  in  (3.6)  after  every  single  observation  and  this  can  be  tedious.  How¬ 
ever,  it  is  possible  to  use  (3.6)  to  construct  a  set  of  stopping  points  for  any  given 
P*.  This  set  turns  out  to  be  fairly  small  and  thus  becomes  a  convenient  method 
of  describing  the  entire  stopping  rule.  Illustrations  of  such  stopping  sets  are 
given  in  Table  I  for  A*  —  0.1,  0.2  and  P*  =  0.75,  0.90,  0.95,  0.99.  For  example, 
for  the  pair  (A*  =  0.2,  P*  =  0.90)  there  are  11  pairs  of  stopping  points  given  in 
Table  I.  If  F\  =  F2  =  F»,  then  sampling  terminates  as  soon  as  T2  =  Sz  —  S2  ^  10 
and  Ti  =  Si  —  Si  ^  26  or  as  soon  as  T2  ^  11  and  Ti  ^  17  or,  and  so  forth. 

A  conservative  variation  of  the  above  rule  replaces  (1  —  p)/(  1  —  p  +  A*)  in 
(3.6)  by  its  upper  bound  1  when  Fz  ^  min  (Fi,  F2)  and  we  obtain  the  following 
stopping  rule. 


TABLE  I 


Stopping  Points  for  the  Sequential  Likelihood  Procedure 
Rlpvt  with  k  =  3  Populations 
Ti  =  St  —  Si,  Tt  =  S3  —  St  as  defined  in  the  text. 

Note:  some  numerical  results  for  these  procedures  are  given  in  Tables  V,  VI,  and  VII. 


A*  =  0.1 


Fi  —  F% 

=  Fs 

II 

=  F3  +  1 

Ft  -  Ft 

=  Fi  -  1 

Fi  =  Fi 

-  Ft  -  1 

p* 

Ti 

Tt 

Ti 

Tt 

Ti 

Tt 

Ti 

Tt 

0.75 

18 

17 

11 

10 

13 

13 

13 

13 

19 

16 

13 

9 

15 

12 

14 

11 

20 

15 

14 

8 

20 

11 

15 

10 

22 

14 

17 

7 

16 

9 

25 

13 

23 

6 

18 

8 

29 

12 

20 

7 

38 

11 

26 

6 

0.90 

28 

27 

20 

19 

23 

23 

23 

21 

30 

26 

21 

18 

24 

22 

24 

20 

31 

25 

22 

17 

30 

21 

25 

19 

33 

24 

24 

16 

26 

18 

37 

23 

28 

15 

27 

17 

42 

22 

36 

14 

29 

16 

61 

21 

32 

15 

40 

14 

0.95 

35 

35 

26 

25 

30 

29 

30 

27 

36 

34 

27 

24 

37 

28 

31 

26 

37 

33 

29 

23 

32 

25 

38 

32 

31 

22 

33 

24 

41 

31 

34 

21 

34 

23 

44 

30 

42 

20 

36 

22 

50 

29 

39 

21 

78 

28 

47 

20 

0.99 

51 

50 

40 

39 

45 

44 

45 

42 

52 

49 

41 

38 

46 

40 

54 

48 

42 

37 

47 

39 

56 

47 

44 

36 

48 

38 

58 

46 

47 

35 

49 

37 

63 

45 

53 

34 

51 

36 

75 

44 

54 

35 

59 

34 

A* 

=  0.2 

Fi  —  Ft 

-  Ft 

Fi  =  Ft 

=  Fi  +  1 

Ft  —  Fi 

=  Fi  -  1 

F>  =  Ft 

=  Ft  -  1 

p* 

Tx 

T2 

Ti 

Tt 

Ti 

Tt 

Ti 

Tt 

0.75 

9 

8 

4 

4 

6 

6 

6 

5 

10 

7 

6 

3 

9 

5 

7 

4 

12 

6 

10 

2 

8 

3 

24 

5 

11 

2 

0.90 

13 

13 

8 

8 

11 

10 

11 

8 

15 

12 

9 

7 

12 

7 

17 

11 

11 

6 

14 

6 

26 

10 

0.95 

17 

16 

11 

10 

14 

14 

14 

11 

19 

15 

13 

9 

15 

10 

22 

14 

20 

8 

17 

9 

23 

8 

0.99 

24 

24 

17 

17 

21 

21 

21 

18 

25 

23 

18 

16 

22 

17 

27 

22 

20 

15  .  ' 

23 

16 

32 

21 

24 

15 
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Stopping  rule  for  the  conservative  likelihood  procedure  jRlpw* 
soon  as  Fs  ^  min  (Fh  F2)  and 


(3.7) 


(1  -  A*)T'  +  (1  -  A*)r»  ^ 


1  -  P* 

p* 


stop  sampling  as 


It  will  be  seen  in  Section  7  that  when  P*  =  0.95  and  A*  =  0.2,  this  conserva¬ 
tive  rule  roughly  causes  a  20  per  cent  increase  in  the  total  expected  number  of 
observations,  E{N}  above  that  for  the  procedure  .ft lpw. 

3.2.  Likelihood  rule  for  PW  sampling  with  k  =  2.  The  special  case  k  f  2  is 
of  particular  interest  because  we  can  make  the  procedure  more  explicit  and  be¬ 
cause  we  can  make  comparisons  with  other  procedures  already  studied,  for  exam¬ 
ple,  the  procedure  ftpw  in  [10].  The  derivation  in  Section  3.1  above  gives  for 
k  =  2  the  following  stopping  rule. 

Stopping  rule  for  ftLPw(&  =  2):  stop  as  soon  as  F2  ^  F\  (that  is,  F\  —  F2  =  0 
or  1)  and 


(3.8) 


a{py=r(r^n« 


max 

A* 


1  -  P* 

p* 


After  randomization,  let  I  denote  the  population  that  we  sample  from  first 
and  II  the  other  population.  Then  Fi  =  F2  in  (3.8)  when  we  are  sampling  from 
I  and  Fi  =  F2  +  1  in  (3.8)  when  we  are  sampling  from  II  and  II  has  more 
successes.  Hence,  we  stop  and  select  I  as  soon  as  S2  —  &  =  t  (if  this  happens 
before  another  equality  below),  where  t  >  0  is  the  smallest  integer  equal  to  or 
greater  than  the  solution  of 


(3.9) 


log  (1  -  A*) 


We  stop  and  select  II  as  soon  as  S2  —  Si  =  s  (if  this  happens  first),  where  s  >  0 
is  the  smallest  integer  for  which 


(3.10) 


Siftv-TG-1,;*.)}* 


max 

A* 


1  -  P* 

p* 


It  is  easily  seen  that  t  ^  s  and  that  we  will  only  select  a  population  after  getting 
a  success  from  that  same  population.  This  differs  from  the  procedure  ftpw  in  [10] 
only  in  that  we  allow  t  ^  s  and  in  [10]  only  t  =  s  is  considered  (see  Table  II). 

Using  the  recursion  formula  method  given  in  [10],  we  can  now  derive  an  exact 
expression  for  the  ft(CS),  E{N},  and  the  expected  number  of  observations 
E{Nb}  on  the  poorer  treatment.  This  will  be  done  for  arbitrary  positive  s  and  t 
and,  as  a  special  case,  we  can  then  set  s  and  t  equal  to  the  values  obtained  above 
by  the  likelihood  approach.  Let  p  (respectively,  p')  be  associated  with  popula¬ 
tion  I  (respectively,  II),  let  NT  =  I  mean  that  the  next  trial  is  on  population  I, 
let  (s,  t)  denote  the  stopping  points,  and  define 

.  .  P„  =  P„(s,  t)  =  P  {I  is  selected |$z  —  Sn  =  n,  NT  =  I,  (s,  t)}, 

‘  Qn  =  Qn(s,  t)  =  P  {I  is  selected]#*  —  Sn  =  n,  NT  =  II,  (s,  t)}. 
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Stopping  Points  for  the  Sequential  Likelihood  Procedure 
2?lpw  with  k  *=  2  Populations 


A* 

=  0.1 

A* 

=  0.2 

p* 

Pi  =  Pi 

Pi  =  P2  +  1 

Pi  =  Fi 

Pi  =  Fi  +  1 

si-sl 

Si -Si 

Si -Si 

S2  —  Si 

0.75 

11 

6 

5 

2 

0.90 

21 

14 

10 

6 

0.95 

28 

20 

14 

8 

0.99 

44 

34 

21 

15 

The  Common  Values  of  r  Required  by  the  Procedure 
Ppw  of  [10] 

Use  7  and  8  with  probability  (weight)  0.679  and  0.321,  respec¬ 
tively,  in  order  to  obtain  a  PCS  of  exactly  P*  =  0.75 
with  A*  =  0.1. 


P* 

A*  -  0.1 

A*  =  0.2 

0.75 

7.32 

3.19 

0.90 

16.45 

7.38 

0.95 

22.96 

10.44 

0.99 

37.82 

17.56 

Then  the  PW  sampling  scheme,  conditional  on  7  being  the  better  population 
leads  to  the  recursion 


Pn  =  Vpn+ 1  +  qQ 

Qn  =  P'Qn- 1  +  q'P  n, 


with  boundary  conditions  Pt  —  1  and  Q-»  =  0. 
From  (3.12),  we  find  that 

(3.13)  P„(S,  i)  -  I  Q*(s,  0 


q'(  1  -  \a+n ) 
q'  —  q\t+t  * 


where  X  =  p'/p  1.  Setting  n  =  0,  we  obtain  the  conditional  PCS  =  Po(s,t ) 
given  that  J  is  the  better  population.  For  the  same  problem,  we  define  the  dual 
expressions 

.  .v  P'n  =  P»(s,  t)  =  P  {II  is  selectedl&T  —  &  =  n,  NT  =  77,  (s,  t)}, 

Q'n  =  q;(s,  t)  =  P  {77  is  selectedjiSfr  —  Si  -  n,  NT  =  7,  (s,  0}, 

and  let  p  (respectively,  p')  be  associated  with  77  (respectively,  7).  Then  we  find 
that  the  recursive  scheme  is  exactly  as  in  (3.12)  with  the  new  boundary  condi¬ 
tions  Pi  =  1  and  Q'~t  =  0,  which  differ  from  the  above  only  in  that  s  and  t  are 
interchanged.  It  fellows  that  the  conditional  PCS  given  that  77  is  the  better 
population  is  Qo(s,  t)  and  this  is  obtained  from  (3.13)  by  merely  interchanging 
s  and  t,  that  is,  Qo(s,  t )  =  Qo(t,  s ).  Hence,  from  these  two  conditional  PCS  re¬ 
sults,  we  obtain 
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(3.15) 


P  rpoip  ■>  Po(s,  t)  +  Qo(t,  s )  q’  -  \(q\*  +  q'\l) 

r,{C-b|/tLpw}  =  2  ~  q 7  -  q\°+< 


It  is  easily  seen  that  for  the  three  extreme  cases  p'  — >  0,  p'  — *  p  >  0,  and  p  — >  1 
we  obtain  from  (3.15),  P(CS)  =  1,  3^,  and  1  —  {pr)l/2,  respectively. 

Using  an  analogous  method  to  obtain  E{Nb},  the  number  of  observations  on 
the  poorer  population,  we  define 


Un  =  Un(s,  t )  =  E{Nn\S z  -  S„  -  n,  NT  =  I,  (s,  <)}, 

Vn  -  7,(8,  0  =  EWhISt  -  Sn  =  n,  NT  =  II,  («,  <)}, 

and  associate  p  with  population  /.  Then  the  PW  sampling  scheme  leads  to 
the  recursion 

U n  =  pU n+l  “b  n> 

Fn  =  p'Pn-l  +  g'Un  +  1, 


(3.16) 


(3.17) 


with  boundary  conditions  Ut  =  0  =  F_,.  It  can  be  shown  (and  it  is  sufficient  to 
verify)  that 

77  (Q  ft  -  ~  n)  _  g[p  +  -  XQ 

(3  18)  nl  ,  ;"pa-X)  p(l  -  X)(g'  -  gX‘+0  ’ 

V  (o  ft  —  P  +  g(*  ~  ft)  _  [p  +  g(*  +  OMfl'X"  -  gXQ 
p(l-X)  p(l  -  X)(g'  -  3X-+‘) 

The  conditional  E{Nb}  given  that  I  is  the  better  population  is  V0(s,  t).We  again 
define  new  quantities  dual  to  (3.16)  by  writing 

„  1Qs  U'n  -  U'n(s,  t )  =  £^{AT/|Sii  -  Sj  =  n,  NT  =  II,  (a,  t)}, 

^.iy;  v,  =  7/(8>  0  =  £!{iV/|<S/7  —  Si  —  n,  NT  —  /,  (a,  *)}, 


and  letting  p  be  associated  with  population  II.  Then  we  get  the  same  recursion 
scheme  as  in  (3.17)  with  the  new  boundary  conditions  U',  =  0  =  Vr-t,  so  that  we 
need  only  interchange  s  and  t  in  (3.18)  to  solve  for  (3.19).  Hence,  the  conditional 
E{Nb }  given  that  II  is  the  better  population  is  Fo(s,  t)  =  V0(t,  s ).  Hence,  from 
(3.18), 


(3.20) 


E  {Nb\Rlvw}  = 


Uojs,  t )  +  Vo(t,  a )  [p  +  q(s  +  OKI  -  XQ(g'  -  gX») 

2  2p(l  -  X)(g'  -  q\*+ *) 


Similarly,  to  find  ,  we  replace  iV/j  by  Ni  in  (3.16)  and  obtain,  in  place 

of  (3.17), 


(3.21) 


Un  =  pUn+1  +  +  1, 

fn  =  p'Fn-1  +  q'Un, 


with  boundary  conditions  Ut  =  0  =  F_s.  The  solution  of  this  set  is 


77  U  ft  -  jit  ~  _  g[P"  +  +  *)MX"  ~  X‘) 

Un^ “  p(  1  -  X)  p(l  -  X)(9'  -  <zX‘+0 


«  /  A  _  p'  +  q'jt  -  n)  _  | y  +  g'(s  +  Q]X*(g'Xw  -  qV) 
Vn t}  ~  p(  1  -  X)  p(l  -  X)(9'  -  gX»+‘) 


(3.22) 
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Again,  we  set  up  the  dual  quantities 

(O  9oN  ft  -  ftfo  0  =  —  Si  =  n,  NT  =  7/,  («,  <)}, 

'  '  ft  =  ft(»,  <)  =  S{iV„|S„  -  Sr  =  n,  NT  =  7,  (s,  ()}, 

and  let  p  be  associated  with  population  77.  Then  the  recursion  is  the  same  as 
in  (3.21)  with  the  boundary  conditions  Ui  =  0  =  V'-t,  so  that  we  need  only 
interchange  s  and  t  in  (3.22).  Hence,  the  conditional  E{NA]  given  that  77  is  the 
better  population  is  Vo(s,  t)  =  V(t,  s ).  It  follows  from  (3.22)  that 


(3.24)  P{A,i|Plpw}  = 


Uo(s,  t )  +  Fp (t,  s)  [p/  +  q'(s  +  Q](g'  -  gX»)(  1  —  XQ 
2  2p(l  -  \)(q'  -  q\*+t) 


Adding  E{NA}  and  E{Nb},  gives 


(3.25)  7?  {Ar|7?LPw}  =  f-±f  +  t)){'r=fj(fzr^r)’ 
where  p  =  (p  +  p')/2  and  q  =  1  —  p. 

For  the  case  p  =  p',  we  take  the  limits  in  (3.20),  (3.24),  and  (3.25)  as  p'  — » p 
and  obtain 

(3.26)  E{N„\R^}  =  E{Na\Rl™}  =  =  <(p^g8). 


which  is  comparable  with  r(p  +  qr)/2p  obtained  in  (2.11)  in  [11]  for  the  pro¬ 
cedure  7W  with  s  =  t  (=  r).  If,  as  is  usually  the  case,  we  have  t>  r>  s  and 
st  <  r2,  then  each  of  these  three  expectations  is  smaller  under  7W  for  q  close  to 
zero  and  each  is  smaller  under  JRlfw  for  p  close  to  zero.  Thus,  neither  of  these 
procedures  can  be  uniformly  better  than  the  other,  that  is,  throughout  the 
parameter  space.  Since  t  in  (3.9)  is  asymptotically  (A*— >0)  like  r  in  (2.13)  of 
[11],  it  follows  that  the  same  lack  of  a  uniform  result  holds  in  comparing  i?LPw 
and  the  vector  at  a  time  procedure  R\t,  that  is,  E{N\RyT}  is  smaller  for  p  — » 0 
and  E{N\Rlpw}  is  smaller  for  p  — *  1. 

To  illustrate  the  results  of  procedure  #lpw  and  compare  them  with  the  pro¬ 
cedure  Rpvr  in  [10],  we  consider  the  pair  (P*  =  0.95,  A*  =  0.2)  and  put  the 
results  in  tabular  form.  For  the  procedure  Ppw,  we  need  to  randomize  between 
r  =  10  (with  probability  0.555)  and  r  =  11  (with  probability  0.445);  this 
achieves  the  P*  value  0.555(0.945)  +  0.445(0.956)  =  0.950  in  the  LF  configu¬ 
ration.  For  the  procedure  Plpw,  we  randomize  between  the  pair  (s  =  7,  t  =  11) 
with  probability  0.434  and  the  pair  (s  =  8,  t  =  12)  with  probability  0.566;  this 
achieves  the  P*  value  0.434(0.943)  +  0.566(0.955)  =  0.950  in  the  LF  configu¬ 
ration.  In  randomizing  between  these  two  particular  pairs  (7, 11)  and  (8, 12), 
for  procedure  Plpw,  rather  than  other  pairs,  such  as  (7,  12)  and  (8, 12),  our 
criterion  was  to  minimize  the  maximum  of  E{Nb},  which  generally  occurs  at 
p  =  A*/2.  This  also  seems  to  minimize  the  maximum  for  E{N}  and  E{NA}, 
which  also  generally  occur  at  p  =  A */2. 

The  comparison  of  Ppw  and  Plpw  in  Table  III  shows  that  the  latter  has  a 
smaller  E{Nb}  and  E{N}  in  17  out  of  the  18  entries.  Thus,  we  have  effected  a 
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TABLE  III 

A  Comparison  of  Procedures  Rpw  and  JElpw  for  k  =  2,  P*  =  0.95,  and 
A*  =  0.2  in  the  GLF  Configuration  A  =  0.2 
Note  that  randomization  was  used  to  make  P* '  —  0.95  exactly  in  both  cases; 

see  text  for  details. 


.  V  +P' 
P==  2 

E\Nb\ 

f?pw  -Rlpw 

E[Na\ 

Spw  1?LPW 

E{N\ 

R?w  Rlpvt 

0.1 

42.28 

38.76 

52.22 

47.83 

94.50 

86.59 

0.2 

37.31 

34.22 

47.25 

43.29 

84.55 

77.51 

0.3 

32.29 

29.56 

42.22 

38.59 

74.51 

68.15 

0.4 

27.13 

24.71 

36.99 

33.61 

64.12 

58.33 

0.5 

21.85 

19.80 

31.55 

28.51 

53.40 

48.31 

0.6 

16.60 

15.04 

26.08 

23.50 

42.68 

38.54 

0.7 

11.55 

10.54 

20.77 

18.80 

32.32 

29.33 

0.8 

6.77 

6.33 

15.79 

14.50 

22.56 

20.83 

0.9 

2.26 

2.31 

11.23 

10.69 

13.49 

13.00 

fairly  uniform  improvement,  with  emphasis  on  the  maximum  value  at  p  =  A*/2, 
although  (as  was  expected)  the  improvement  is  not  substantial  anywhere.  How¬ 
ever,  in  the  context  of  clinical  trials  even  slight  decreases  in  E  {NB}  are  important. 


4.  An  elimination  procedure  2?epw 


For  k  >  2,  we  define  an  elimination  procedure  which  is  an  extension  of  the 
procedure  J F«W  defined  for  k  =  2  and  studied  in  [10].  Under  PPW,  we  stop  sam¬ 
pling  when  |  s*  —  Sy|  =  r,  where  s,  is  the  number  of  successes  from  xf,  i  ^  j, 
i  =  1,  2,  j  =  1,  2.  Assuming  s,-  >  Sy,  we  then  select  xt-  as  the  better  population. 
The  approximate  value  of  r  required  to  satisfy  (2.1)  is  the  smallest  integer  equal 
to  or  greater  than  r2,  where 


—  log  2(1  —  P*) 
log  (1  - 


We  extend  this  procedure  as  follows.  Population  xy  is  eliminated  if  for  some  x< 
(not  yet  eliminated)  —  sy  =  r.  Let  x*  be  the  best  population.  Since 


(4.2)  1  —  P{CS}  ^  2P  {xi  eliminates  x*}  ^  (k  —  1)(1  —  JP*), 

it  follows  that 


(4.3)  P{CS}  ^  1  -  (k-  1)(1  -  P*). 

If  we  now  set  the  right  side  of  (4.3)  equal  to  P*,  solve  for  P*,  and  substitute 
the  result  in  (4.1),  then  it  is  clear  that  the  resulting  procedure  which  uses 
throughout  for  r  the  smallest  integer  equal  to  or  greater  than 


logi  2 


{ 


a  -p*n 

►  k  —  1  J 


log  (1  -  A*) 


(4.4) 


rk  = 
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satisfies 

(4.5)  P{CS|flEpw}  ^  P*  whenever  A  ^  A*. 

Monte  Carlo  results  for  Pepw  and  comparisons  with  other  procedures  can  be 
found  in  Tables  V,  VI,  and  VII  (below). 

6.  Elimination  with  Wald’s  double  dichotomy 

For  ft  ^  2,  we  investigate  the  numerical  results  of  an  elimination  procedure 
which  is  derived  in  [5]  and  based  on  general  methods  from  [3]  applied  to  the 
double  dichotomy  problem  as  formulated  by  Wald  [14]. 

This  procedure  jKevt  uses  the  VT  sampling  rule  and  eliminates  population  7 r> 
if  for  some  ir<  (not  yet  eliminated), 

(5.1)  Si  —  Sj  t:  c  +  dn*, 

where  c  >  0  and  d  ^  0  are  predetermined  constants  and  n*  is  the  number  of 
unlike  pairs  from  7r<  and  717  (that  is,  observations  in  the  same  vector  of  the  form 
S,  F  or  F,  S). 

We  now  give  the  values  of  c  and  d  that  satisfy  the  requirement  (2.1).  Define 
r0  by 

(5.2)  T“ =  (nFr*y  <  h 

and  let  n  denote  any  value  such  that 

(5.3)  TO  <  Tl  S  — ' 

TO 


It  is  shown  in  [5]  that  by  taking 


the  requirement  (2.1)  will  be  satisfied.  We  select  n  =  1/to  for  our  Monte 
Carlo  studies  and  this  implies  that  d  =  0,  the  reason  being  that  asymptotically 
(P*  — »  1)  at  the  generalized  least  favorable  configuration  (that  is,  when  p[ u  = 
Pi 2]  =  •  •  •  =  p[*_i]  =  p[k\  —  A*),  the  risk  defined  in  (2.3)  is  minimized  for  this 
value  of  n  (see  [5]). 

The  use  of  d  =  0  above  also  provides  us  with  the  analogous  elimination  pro¬ 
cedure  for  extending  the  procedure  Pvt  in  [10]  to  k  >  2  in  the  same  way  that 
we  extended  Rfw  in  Section  4. 


6.  Comparisons  of  several  play  the  winner  procedures  for  ft  =  2 

In  this  section,  our  aim  is  to  make  a  comparison  for  ft  =  2  of  the  likelihood 
procedure  developed  in  Section  3.2  with  some  other  procedures  that  satisfy  the 
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TABLE  IV 

Expected  Sample  Sizes  for  k  -  2  Under  Five  PW  Procedures  (P*  =  0.95,  A*  =  0.2) 
The  table  gives  E{Nb),  E{N]  for  A  =  0.2  and  E0[N]  for  A  =  0  in  each  cell. 

For  Rj  and  Rio,  use  r  =  20  and  21  with  weights  0.958  and  0.042,  respectively. 

For  Rh,  use  r  =  33  and  34  with  weights  0.6  and  0.4,  respectively. 

For  Rit,  use  r  =  20  and  21  with  weights  0.761  and  0.239,  respectively. 


_  p  +p' 
2 

Klpw 

Ri 

Rio 

Rh 

Rit 

0 

£o{V) 

00 

00 

00 

65.8 

40.5 

E{Nb\ 

38.8 

26.8 

20.2 

0.1 

E{N} 

86.6 

180.9 

59.7 

45.5 

E0[N\ 

348.4 

64.2 

45.0 

E[Nb } 

34.2 

52.5 

26.1 

22.5 

0.2 

E{N\ 

77.5 

119.3 

118.8 

51.4 

E0{N } 

362.6 

172.5 

63.2 

50.6 

E{Nb\ 

29.6 

38.2 

37.5 

25.2 

24.9 

0.3 

E{N } 

68.2 

88.3 

87.6 

58.0 

Eo[N} 

216.4 

114.4 

113.7 

62.2 

57.8 

E{Nb } 

24.7 

29.2 

28.5 

24.1 

25.7 

0.4 

E{N] 

58.3 

69.1 

68.4 

56.9 

61.1 

Eo{N\ 

143.2 

84.8 

84.2 

61.4 

65.3 

E{Nb\ 

19.8 

22.9 

22.1 

22.6 

22.7 

0.5 

E{N } 

48.3 

55.2 

55.3 

55.7 

Eo{N] 

99.4 

66.2 

64.8 

E{Nb } 

17.7 

18.0 

0.6 

P{iV) 

38.5 

45.2 

53.2 

46.6 

Eo{N) 

54.8 

53.9 

55.3 

e\Nb ) 

10.5 

13.5 

12.5 

17.5 

13.6 

0.7 

E[N\ 

29.3 

38.2 

37.1 

50.2 

38.5 

EoiN] 

49.2 

45.8 

44.7 

57.6 

46.3 

E{Nb\ 

6.3 

8.9 

7.8 

12.5 

8.9 

0.8 

E{N\ 

29.7 

45.3 

31.1 

Eo{N\ 

33.6 

38.4 

37.1 

55.2 

38.9 

E{Nb\ 

2.3 

2.5 

2.5 

2.5 

2.5 

0.9 

E{N } 

22.4 

22.4 

35.4 

22.6 

E0\N\ 

21.4 

31.4 

31.8 

1.0 

E0{N } 

11.6 

33.4 

same  probability  requirement  (2.1)  with  A*  =  0.2  and  P*  =  0.95.  All  of  our 
numerical  entries  for  k  =  2  (in  Table  IV)  are  based  on  exact  formulas.  In  Table 
IV,  we  have  included  in  each  cell  E{Nb]  for  the  LF  configuration  (A  =  0.2)  and 
E{N}  for  the  LF  (A  =  0.2)  and  equal  parameter  (EP)  configuration  (A  =  0). 
The  procedures  Ri  and  Rio  are  inverse  sampling  procedures  using  PW  sampling 
without  and  with  reordering  after  each  complete  cycle,  respectively. 

The  modified  procedure  Rh  due  to  Hoel  [3]  uses  PW  sampling  and  scores, 
where  the  score  WA  (for  drug  A,  say)  is  defined  by  adding  the  successes  of  drug 
A  and  the  failures  of  drug  B  and  the  termination  rule  is  inverse  sampling,  that 
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is,  stop  when  max  {WA,  WB)  =  r.  It  has  a  bounded  E{N}  value  for  p  =  p'  =  0 
and  is  therefore  an  improvement  on  Ri  for  small  values  of  p. 

Another  procedure  Rit,  due  to  Berry  and  Sobel  [2],  modifies  the  inverse  sam¬ 
pling  scheme  by  terminating  the  procedure  either  after  a  fixed  number  c  of 
complete  cycles  or  after  one  population  reaches  r  successes,  whichever  occurs 
sooner.  This  procedure  appears  to  have  two  preassigned  constants  (r,  c)  to  spec¬ 
ify,  but  both  constants  are  used  (with  r  —  c)  to  satisfy  (2.1). 

Table  IV  (for  k  —  2)  shows  that  for  p  =  (p  +  p')/2  >  Yi  the  likelihood  pro¬ 
cedure  is  preferable  using  either  the  risk  criterion  or  E{N}.  However,  for  p  <  Yi 
the  value  of  E  (iV)  becomes  infinite  for  all  three  of  the  procedures,  #lpw,  Ri,  and 
Rio  when  p  =  p' .  Procedures  Rh  and  Rit,  on  the  other  hand,  have  a  bounded 
E{N}  function  even  for  p  =  p'  and  the  numerical  improvement  for  small  p  in 
Table  IV,  especially  for  p  =  p',  is  very  striking.  It  follows,  as  in  the  case  of 
k  —  3  in  the  next  section,  that  if  we  had  some  a  priori  knowledge  about  the 
value  of  p ,  we  could  more  easily  decide  which  of  these  procedures  to  use. 

Procedure  Rio  shows  only  a  small  improvement  over  procedure  Ri ,  but  it  is 
uniform  over  the  entire  parameter  space. 

7.  Monte  Carlo  simulation  studies  for  k  —  3 

In  this  section,  we  bring  together  several  procedures  appropriate  for  k  =  3 
populations  and  make  some  Monte  Carlo  studies  to  compare  them.  The  criteria 
for  comparison  are  the  risk  function  (2.3)  and  the  expected  total  number  of  obser¬ 
vations  E{N}.  The  same  formulation  (2.1)  applies  to  all  these  procedures  with 
the  common  values  P*  =  0.95  and  A*  —  0.2.  Each  entry  in  Table  V  corresponds 
to  the  average  of  the  results  of  1,000  experiments. 

The  main  breakdown  is  between  the  procedures  that  use  PW  sampling  and 
those  that  use  VT  sampling.  We  have  included  three  previously  published  pro¬ 
cedures.  In  the  PW  group,  we  include  the  inverse  sampling  procedure  Ri  studied 
in  [13].  In  the  VT  group,  we  include  the  procedure  Rkks  which  was  developed 
in  [1],  by  Bechhofer,  Kiefer,  and  Sobel,  for  general  k,  but  details  of  which  are 
given  in  ([1],  p.  270)  only  for  k  =  2.  We  also  include  in  the  VT  group  the  pro¬ 
cedure  Rp  due  to  Paulson  [7].  A  brief  description  of  these  procedures  now  follows. 

Under  procedure  Ri,  we  sample  cyclically  from  three  populations  with  PW 
sampling  until  any  one  of  them  has  r  successes;  it  is  then  selected  to  be  the  best 
population.  The  Monte  Carlo  results  for  Ri  given  in  Tables  V,  VI,  and  VII  are 
very  close  to  approximate  values  based  on  (3.35)  and  (3.37)  in  [13].  A  table  of 
these  approximate  values,  not  included  here,  gives  values  consistently  smaller 
than  the  observed  values  in  Table  IV. 

Under  procedure  .Rbks,  we  use  vector  sampling  and  stop  as  soon  as 


and  select  the  population  associated  with  $3.  For  P*  >  Yi,  we  will  not  stop 
when  S3  =  S2,  and  hence,  randomization  will  not  be  required  at  termination.  It 
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TABLE  VI 

Expected  Total  Number  of  Observations  for  Various  Procedures 
k  =  3,  A*  -  0.2,  and  P*  =  0.95 
GLF  configurations  with  1,000  experiments  per  point. 


max  pi 

Play  the  winner  sampling 
jRepw  Ri  Rlpw  ■Rlpw 

Sobel- 

Weiss  Likelihood 

(elimi-  Inverse  conserva-  Likeli- 

nation)  sampling  tive  hood 

Vector  sampling 
■Rbks  Rp  .Revt 

Wald’s 
Paulson  double 

(elimi-  dichotomy 

(see  [1])  nation)  (elimination) 

.20 

184.7 

368.5 

223.3 

184.8 

74.6 

81.1 

74.6 

.25 

177.9 

291.1 

210.3 

171.2 

77.1 

81.0 

.30 

166.5 

238.4 

196.6 

164.1 

77.2 

80.6 

75.8 

.35 

156.4 

204.1 

188.2 

155.5 

77.6 

81.7 

75.8 

.40 

149.6 

176.9 

178.8 

145.9 

76.9 

79.8 

75.8 

.45 

138.8 

153.6 

162.5 

134.5 

77.9 

79.4 

75.4 

.50 

131.6 

137.3 

153.5 

126.1 

78.0 

81.2 

74.6 

.55 

118.3 

123.1 

141.8 

115.1 

76.5 

80.0 

74.8 

.60 

109.2 

109.6 

126.8 

103.0 

76.7 

80.2 

75.2 

.65 

97.2 

99.2 

116.3 

94.5 

77.4 

82.0 

74.6 

.70 

84.5 

89.0 

102.9 

83.0 

78.3 

80.8 

74.8 

.75 

73.4 

80.9 

90.3 

74.7 

81.3 

80.6 

76.9 

.80 

62.3 

71.8 

75.9 

61.4 

80.2 

80.2 

77.4 

.85 

52.1 

63.1 

62.4 

51.0 

81.6 

81.0 

79.1 

.90 

42.9 

54.3 

49.6 

39.3 

80.9 

81.5 

78.9 

.95 

31.4 

44.3 

36.5 

30.2 

80.4 

81.7 

78.8 

1.00 

21.2 

33.4 

23.7 

20.2 

80.1 

81.3 

79.7 

function,  and  Table  VII  gives  the  estimated  PCS  function  (or  observed  fre¬ 
quency  of  success). 

As  a  group,  the  PW  sampling  procedures  are  different  from  the  group  of  VT 
sampling  procedures.  The  latter  procedures  have  remarkably  constant  risk  and 
E{N}  for  varying  values  of  max  i  =  1,  2,  3  while  the  former  procedures  ap¬ 
pear  to  be  monotonically  decreasing  with  max  p^  the  cross  over  point  is  about 
0.65  in  Table  V  and  about  0.75  in  Table  VI.  It  follows  that  if  we  had  some  prior 
knowledge  about  max  pi  (only),  we  might  be  better  able  to  decide  which  type 
of  sampling  to  use. 

Among  the  PW  sampling  rules,  the  procedures  jRepw  and  jBlpw  are  quite 
similar  and  uniformly  better  than  both  the  procedures  Ri  and  the  conservative 
likelihood  procedure.  However,  from  Table  VI,  it  appears  that  the  procedure 
Rlpw  is  slightly  better  than  #epw. 

For  the  VT  sampling  procedures,  the  procedure  Revt  is  preferable  to  both 
.Rbks  and  Rp  using  either  the  risk  or  the  E {N}  criterion;  the  differences  between 
the  latter  two  procedures  appear  to  be  small. 

In  Table  VII  we  note,  as  expected,  that  all  procedures  satisfied  the  require¬ 
ment  (2.1)  in  all  the  experiments  that  were  carried  out.  The  PW  procedures, 
except  for  conservative  likelihood,  came  closer  to  the  nominal  value  P*  =  0.95 
than  the  VT  procedures,  and  hence,  were  slightly  more  efficient  in  the  sense  that 
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TABLE  VII 

Pbob ability  op  Correct  Selection  for  Various  Procedures 
k  =  3,  A*  =  0.2,  and  P*  =  0.95 
GLF  configurations  with  1,000  experiments  per  point. 


max  pi 

Play  the  winner  sampling 

Repw  Ri  Rlpw  R  lpw 

Sobel- 

Weiss  Likelihood 

(elimi-  Inverse  conserva-  Likeli- 

nation)  sampling  tive  hood 

Vector  sampling 

Rbkb  Rp  Revt 

Wald’s 
Paulson  double 

(elimi-  dichotomy 

(see  [1])  nation)  (elimination) 

.20 

1.000 

1.000 

1.000 

1.000 

1.000 

.968 

1.000 

.25 

1.000 

1.000 

1.000 

1.000 

1.000 

.968 

1.000 

.30 

1.000 

1.000 

1.000 

1.000 

.998 

.966 

.999 

.35 

1.000 

1.000 

1.000 

1.000 

.993 

.973 

.993 

.40 

1.000 

.998 

1.000 

1.000 

.984 

.971 

.993 

.45 

1.000 

.991 

1.000 

.997 

.984 

.960 

.989 

.50 

1.000 

.984 

1.000 

.999 

.980 

.970 

.982 

.55 

.996 

.983 

.997 

.990 

.957 

.971 

.974 

.60 

.991 

.967 

.997 

.987 

.961 

.961 

.970 

.65 

.994 

.970 

.997 

.987 

.964 

.972 

.977 

.70 

.991 

.963 

.995 

.981 

.973 

.976 

.969 

.75 

.972 

.952 

.990 

.976 

.983 

.974 

.981 

.80 

.970 

.957 

.985 

.977 

.983 

.969 

.989 

.85 

.967 

.965 

.987 

.968 

.994 

.970 

.997 

.90 

.968 

.969 

.981 

.962 

.999 

.965 

.997 

.95 

.952 

.986 

.978 

.955 

1.000 

.966 

1.000 

1.00 

.957 

.996 

.989 

.969 

1.000 

.964 

1.000 

they  have  less  “excess  over  the  boundary”.  In  addition,  the  columns  of  Table 
VII  give  some  indication  of  where  the  least  favorable  configuration  is  for  the 
pair  (A*  =  0.2,  P*  —  0.95). 

We  wish  to  point  out  that  we  have  not  observed  the  expected  number  of  stages 
required  for  termination  since  (1)  it  is  not  clearly  defined  for  all  procedures, 
and  (2)  it  is  not  crucial  for  the  application  to  clinical  trials.  It  should  also  be 
pointed  out  that  our  Monte  Carlo  results  are  only  for  GLF  configurations,  where 
Pw  —  Vi 2]  =  Pol  —  0.2.  In  other  configurations,  the  elimination  procedures  are 
even  more  preferable,  because  noncompeting  populations  can  be  eliminated  early. 

0  0  0  0  0 

The  authors  wish  to  thank  Professor,  R.  E.  Bechhofer  of  Cornell  University 
for  his  helpful  comments. 
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